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PREFACE 


This  document  summarizes  results  and  insights  from  a  number  of  RAND 
studies  that  assessed  alternative  strategies  for  making  Army  individual 
training  more  efficient  and  affordable.  Although  the  studies  focus  on 
resident  training  conducted  in  U.S.  Army  schools,  many  of  the  insights 
that  emerged  also  apply  to  civilian  education  and  training. 

The  following  publications  were  drawn  upon  in  this  document: 

R-3848-FMP,  Effectiveness  of  Interactive  Videodisc  in  Army 
Communications  Training,  John  D.  Winkler  and  J.  Michael  Polich, 
1990. 

R-4228-A,  Linking  Future  Training  Concepts  to  Army  Individual  Training 
Programs,  John  D.  Winkler,  Stephen  J.  Kirin,  and  John  S.  Uebersax, 
1992. 

N-3527-A,  The  Army  Military  Occupational  Specialty  Database,  Stephen 
J.  Kirin  and  John  D.  Winkler,  1992. 

R-4224-A,  How  to  Estimate  the  Cost  of  Changes  in  Army  Individual  Skill 
Training,  Susan  Way-Smith,  1993. 

MR-118- A,  Distributed  Training  of  Armor  Officers,  John  D.  Winkler, 
Susan  Way-Smith,  Gary  Moody,  Hilary  Farris,  James  P.  Kahan,  and 
Charles  Donnell,  1993. 

MR-119-A,  Device-Based  Training  of  Armor  Crewmen,  Gary  Moody, 
Susan  Way-Smith,  Hilary  Farris,  John  D.  Winkler,  James  P.  Kahan, 
and  Charles  Donnell,  1993. 

MR-120- A,  Computer-Based  Training  of  Cannon  Fire  Direction 
Specialists,  Hilary  Farris,  William  L.  Spencer,  John  D.  Winkler,  and 
James  P.  Kahan,  1993. 

The  results  should  be  of  interest  to  policymakers  concerned  with  military 
education  and  training  and  to  managers  responsible  for  designing  and 
implementing  training  programs  for  specific  military  specialties.  This 
briefing  was  prepared  in  the  Arroyo  Center's  Manpower  and  Training 
Program,  with  additional  support  from  RAND's  Corporate  Research 
Manager  for  Human  and  Material  Resource  Policy.  The  research 
summarized  here  was  conducted  by  the  Arroyo  Center  and  the  National 
Defense  Research  Institute,  which  are  federally  funded  research  and 
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development  centers  sponsored  respectively  by  the  United  States  Army 
and  by  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  and  the 
defense  agencies. 


SUMMARY 


INTRODUCTION 

Individual  training,  which  prepares  soldiers  to  perform  a  military 
occupation  and  which  occurs  in  classrooms,  on  job  sites,  and  through  self¬ 
development,  is  a  large  and  costly  part  of  Army  operations.  It  requires  an 
extensive  training  base — involving  83,000  trainee  man-years  at  68  Army 
schools — and  it  is  multifaceted — involving  over  300  entry-level 
occupations  and  advanced  training  courses,  many  of  which  are  high  in 
technical  complexity. 

Individual  training  also  carries  a  big  price  tag,  entailing  roughly  10 
percent  ($6  billion)  of  the  Army  budget  in  recent  years.  Its  large  size 
makes  individual  training  a  tempting  target  for  budget  reductions  as  the 
Army  downsizes  and  seeks  to  maintain  its  combat  power.  As  a  result,  the 
Army  must  confront  the  question  of  how  it  can  reduce  training  costs  while 
preserving  effectiveness. 

The  Army  has  proposed  a  number  of  measures  to  reduce  costs,  which 
follow  two  general  approaches:  (1)  shift  training  from  schoolhouses  to  job 
sites,  which  can  be  accomplished  by  shortening  or  consolidating  formal 
courses  and  by  expanding  on-the-job  training  (OJT)  and  self-development 
activities  in  units;  and  (2)  use  training  technologies — e.g.,  training  aids, 
devices,  simulators,  and  simulations  (TADSS) — more  extensively  in  school 
or  at  soldiers'  units  and  home  stations  through  "distance  learning" 
technologies. 

Because  these  changes  could  have  far-reaching  effects  on  training  costs 
and  soldier  proficiency,  they  need  to  be  thoroughly  assessed.  The  Army 
asked  the  Arroyo  Center  to  develop  and  apply  research  methods  to 
identify  the  Army  training  programs  in  which  these  new  training 
approaches  could  be  implemented  and  to  assess  their  cost  and 
effectiveness. 


A  SYSTEMATIC  METHODOLOGY 

Arroyo  Center  researchers  developed  a  four-step  research  method  that 
provides  tools  and  analyses  that,  if  applied  sequentially  and 
systematically,  constitute  an  integrated  method  for  evaluating  training 


approaches:  (1)  match  occupations  to  training  approaches;  (2)  develop 
alternative  courses  based  on  job  requirements;  (3)  assess  costs  and  savings 
of  alternative  courses;  and  (4)  conduct  "clinical  trials"  or  experiments  to 
confirm  effectiveness. 

Several  RAND  studies  applied  this  methodology  in  a  variety  of 
occupations  and  training  courses,  covering  officers  and  enlisted,  combat 
and  civilian-like,  high-tech  and  low-tech,  over  such  areas  as  armor, 
artillery,  communications,  and  maintenance.  These  were  generally 
selected  as  test  cases  from  which  results  could  reasonably  generalize  to 
other  occupations  and  training  courses. 


FINDINGS 

Shifting  Training  from  School  to  Work  Settings 

Army  techniques  for  designing  training  programs  to  support  job 
requirements  work  fairly  well,  as  seen  in  three  courses — the  Army  Officer 
Advanced  Course  (AOAC),  the  introductory  course  for  enlisted  tank 
crewmen,  and  the  introductory  course  for  artillery  fire  direction 
specialists.  Across  the  three  courses,  most  material  needed  to  perform  the 
job,  requiring  training  in  school,  was  trained  in  school;  very  little  material 
could  be  omitted  as  "unnecessary";  and  only  a  limited  amount  of  material 
could  be  shifted  to  units  (less  than  was  estimated  under  the  Army's 
distributed  training  plan). 

The  AOAC  showed  the  most  potential  for  shifting  material  to  units  (25 
percent,  around  four  weeks  of  instruction).  The  analysis  examined  the 
effects  of  shifting  this  material  to  units  by  having  the  armor  officers 
engage  in  "self-study"  using  media  like  computer-based  training  (CBT), 
videotape,  and  traditional  paper  products  before  they  attended  the 
AOAC.  Results  showed  that  costs  and  savings  depend  on  the  capacity  of 
the  units  to  absorb  the  additional  training — that  is,  for  example,  whether 
courseware  and  equipment  already  exist,  whether  facilities  like  post 
learning  centers  are  adequate,  and  whether  additional  manpower  would 
be  needed  to  oversee  the  training.  If  one  assumes  the  units  can  absorb  this 
training — the  assumption  initially  favored  by  TRADOC — the  shift  can 
lower  training  costs.  For  a  one-time  cost  of  $1.7  million,  savings  would 
accrue  quickly,  totaling  $8  million  by  the  end  of  a  five-year  period  (a 
breakeven  point  of  one  year). 

However,  if  one  assumes — as  the  analysis  seems  to  indicate — that 
additional  manpower  and  new  equipment  would  be  needed,  that  learning 
centers  would  need  to  be  remodeled,  and  that  there  would  be  other  costs 
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for  development  and  support,  the  initial  start-up  costs  would  increase 
(from  $1.7  to  $7.2  million).  At  the  same  time,  the  recurring  savings  would 
decrease  considerably  (from  $8  million  in  five  years  to  only  $4  million). 

The  breakeven  point  would  therefore  go  from  one  year  to  nine  years. 

In  addition,  the  payoff  is  sensitive  to  the  mix  of  media  used  to  deliver  the 
training.  The  above  analysis  used  a  mixture  of  CBT,  paper,  and  videotape. 
However,  using  a  more  high-tech  mix  that  included  teletraining  and  other 
media  turned  out  to  cost  more  each  year  to  operate  than  current  training 
under  any  set  of  assumptions.  In  general,  as  distance  learning  media 
become  more  complex — including  both  hardware  requirements  and  the 
range,  quality,  and  fidelity  of  the  courseware — the  one-time  start-up  costs 
increase,  while  the  recurring  savings  decrease. 

Expanding  the  Use  of  Training  Technologies 

In  looking  at  alternative  uses  of  resources  to  train  in  the  schoolhouse,  we 
find  that  use  of  TADSS  can  be  expanded  in  many  courses.  The  Abrams 
armor  crewman  course  provides  a  good  example.  While  the  current 
course  uses  training  technology  (a  driver  simulator)  for  only  11  percent  of 
its  current  course  hours,  an  alternative  course  could  decrease  the  use  of 
tanks  from  59  percent  to  20  percent.  It  would  do  this  by  increasing  the  11 
percent  use  of  training  technology  to  59  percent  by  using  such  training 
devices  as  turret  trainers  and  hull  driver  trainer  vehicles. 

Although  analysis  showed  that  such  technology  applications  could  reduce 
costs,  the  amount  depends  on  how  the  training  devices  are  obtained.  If 
they  are  built  from  scratch — say,  developed  by  a  defense  contractor — 
start-up  costs  of  $20  million  would  return  $16  million  in  the  initial  five- 
year  period,  with  breakeven  in  approximately  six  years.  Given  the 
lifecycle  of  the  Abrams  tank,  this  is  probably  a  worthwhile  investment. 

But  if  the  devices  are  adapted  from  existing  tanks,  by  converting  the  tanks 
in  inventory  at  Fort  Knox  into  turret  trainers  and  hull  driver  vehicles, 
start-up  costs  of  $8  million  would  return  $38  million  in  savings  in  five 
years,  with  breakeven  in  around  two  years.  These  savings  accrue  because 
the  school  would  no  longer  need  to  operate  and  maintain  the  converted 
tanks. 

While  using  training  devices  can  save  money — as  shown  above — the 
effectiveness  of  those  training  devices  depends  on  how  they  are  used. 
Because  the  training  technologies  for  the  armor  crewmen  have  not  been 
built  yet,  the  effectiveness  of  die  technologies  could  not  be  assessed; 
however,  another  series  of  studies  did  examine  this  issue  in  two 
communications  courses  at  the  Army  Signal  School.  The  studies  involved 
experiments  that  used  random  assignment  of  students,  monitoring  of 
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training,  and  hands-on  performance  tests.  They  were  designed  to  test  two 
contrasting  strategies  for  using  training  technology:  (1)  enhance  training 
by  adding  technologies  to  existing  resources;  and  (2)  substitute  for  hands- 
on  training  in  a  resource  mix,  as  in  the  armor  crewman  example. 

In  the  enhancement  experiment,  student  performance  did  not  improve 
significantly.  Researchers  created  an  experimental  classroom  in  a 
multichannel  communications  equipment  operator  course  and  provided 
interactive  videodiscs  (IVDs)  to  enhance  the  training.  In  the  control 
classroom,  training  was  done  exclusively  on  multichannel  radio  sets.  The 
IVD  program  was  fully  accepted  and  implemented  in  the  experimental 
classroom,  and  it  did  increase  practice  time  by  45  percent  within  the  same 
course  length.  However,  enhancement  produced  no  significant 
improvement,  primarily  because  initial  proficiency  was  already  high. 

In  the  substitution  experiment,  IVDs  were  used  in  place  of  more  expensive 
tropospheric  scatter  radios  in  a  tactical  satellite/microwave  systems 
operator  course.  In  this  case,  despite  the  resource  substitution,  the 
students  received  roughly  equivalent  training  in  the  two  classrooms. 

And,  more  important,  students  from  the  two  classrooms  were  equally  able 
to  perform  the  required  tasks  to  Army  standard.  At  the  same  time,  the 
experimental  classroom  cost  one-fifth  as  much  as  the  control  classroom  in 
terms  of  equipment  costs  and  one-half  as  much  in  instructor  costs,  since 
running  the  experimental  classroom  required  less  over-the-shoulder 
supervision. 


ARMY  USES  OF  RESEARCH 

Based  on  the  analysis  discussed  here,  as  well  as  on  a  series  of  other 
studies,  the  researchers  recommended  how  to  restructure  courses,  invest 
in  technology,  and  use  training  resources  most  effectively.  The  Army 
adopted  a  number  of  the  specific  policy  recommendations  and  is  planning 
to  convert  tanks  to  training  devices  as  recommended,  has  dropped  plans 
for  teletraining  equipment  totaling  $180  million,  and  has  restructured 
training  courses  along  the  lines  recommended  in  the  studies. 

In  addition,  the  Army  adapted  and  implemented  several  of  the 
methodologies  used  in  the  research.  Specifically,  the  approach  for 
reviewing  occupations'  job  requirements  and  current  training  approaches 
has  been  incorporated  as  part  of  a  new  Army  occupational  training 
system,  and  the  methodology  developed  for  analyzing  training  resources 
and  costs  has  been  incorporated  in  Army  professional  education  courses 
for  training  developers  and  managers. 
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GENERAL  LESSONS  LEARNED 


First,  the  research  showed  the  need  to  ensure  proper  alignment  between 
school  and  work  by  setting  skill  standards  and  measuring  job  performance, 
and  by  using  job  performance  data  to  develop  and  revise  curricula. 
Second,  the  research  revealed  the  need  to  develop  an  investment  plan  for 
training  technology,  one  that  emphasizes  existing  technologies  (adapted 
as  needed),  one  that  adds  resources  only  where  performance  is  deficient, 
and  one  that  substitutes  technology  in  place  of  more  expensive  training 
resources.  Finally,  the  research  confirmed  the  value  of  field  trials  and  tests  of 
new  training  methods,  which  provide  direct  evidence  of  the  costs  and 
effectiveness  of  new  training  approaches. 

Some  of  these  lessons  also  apply  to  civilian  education  and  training. 
Although  the  two  systems  differ  across  a  number  of  dimensions,  the 
lessons  may  transfer,  especially  in  technical  fields.  Specifically,  the 
military  experience  with  occupational  skills  standards  and  performance 
measures  for  comparable  jobs,  instructional  design  methods  for  aligning 
training  with  work,  and  educational  courseware  and  simulations  may  be 
applicable  to  civilian  settings.  More  important,  the  military  experience 
shows  the  need  and  the  value  of  attending  carefully  to  costs  and  benefits 
and  in  conducting  systematic  assessments  of  education  and  training 
innovations. 
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Improving  Training  at  School  and  Work: 

Lessons  from  RAND  Research 
on  Army  Individual  Training 


This  briefing  summarizes  lessons  from  research  on  military  education  and 
training,  conducted  over  several  years  in  two  of  RAND's  federally  funded 
research  and  development  centers — the  Arroyo  Center  and  the  National 
Defense  Research  Institute. 

The  primary  focus  of  this  research  has  been  on  individual  skill  training, 
which  prepares  soldiers  to  perform  duties  of  a  military  occupation.  Much 
of  this  training  is  done  in  classrooms  at  U.S.  Army  schools,  but  some  of  it 
is  carried  out  on  job  sites  (e.g.,  in  tactical  units  such  as  divisions)  and 
through  self-development  (e.g.,  by  self-study  using  correspondence 
courses  and  audiovisual  materials). 

This  document  synthesizes  the  findings  of  this  research  and  draws  general 
conclusions  and  policy  implications  for  training  in  the  military.  It  also 
highlights  findings  that  are  relevant  to  current  policy  debates  in  civilian 
education  and  training. 
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The  Army  Education  and  Training  System 
Is  Large  and  Costly— and  Must  Come  Down 


•  Extensive — 83,000  trainee  man-years  at  68  Army 
schools 

•  Multifaceted— 300+  occupations,  85%  with 
civilian  counterparts 

•  Expensive— $6  billion  (nearly  10%  of  Army 
budget) 


Problem:  How  to  reduce  costs  but  preserve  effectiveness 


Individual  training  in  the  Army  is  a  big  enterprise.  First,  the  training  base 
is  extensive.  The  portion  conducted  in  Army  schools  involves  large 
numbers  of  personnel  and  large  quantities  of  facilities,  installations, 
equipment,  and  consumables.  Second,  the  training  is  multifaceted, 
involving  a  range  of  occupations  and  training  courses  (over  300  entry- 
level  occupations  and  advanced  training  courses).  Many  of  these 
occupations  are  high  in  technical  complexity  (e.g.,  jobs  in  electronics, 
health  care,  and  repair/maintenance).  In  addition,  much  of  the  skill  base 
is  common  to  the  civilian  sector — 85  percent  of  occupations  have  a  direct 
civilian  counterpart  in  the  U.S.  Department  of  Labor's  Dictionary  of 
Occupational  Titles  (e.g.,  maintenance,  construction). 

This  training  also  has  a  high  price  tag:  roughly  $6  billion,  around  10 
percent  of  the  Army  budget.  This  amount  provides  a  tempting  target  for 
budget  reductions  as  the  military  downsizes  and  seeks  to  maintain  its 
combat  power. 

Thus,  a  key  question  confronting  the  Army  is:  How  can  it  reduce  the  costs 
of  training  while  preserving  training  effectiveness? 
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Army  Efforts  to  Cut  Costs  and  Preserve  1 

Effectiveness  Follow  Two  Approaches  | 

Shifting  training  from  school 

•  Shorten  courses  and  reduce 

to  work  settings 

instructor  manpower 

•  Consolidate  occupations  and 
courses 

•  increase  self-development  and 

OJT  in  units 

Expanding  use  of  training 

•  Conduct  computer-supported 

technologies 

simulations 

•  Use  “distance  learning” 

technologies  in  work  settings 

The  Army  has  proposed  a  number  of  measures  to  reduce  training  costs, 
which  follow  two  general  approaches. 

•  Shift  training  from  schoolhouses  to  job  sites.  This  can  be  accomplished  by 
shortening  or  consolidating  formal  courses,  and  expanding  on-the-job 
training  (OJT)  and  self-development  activities  in  units. 

•  Use  training  technologies,  e.g.,  training  aids,  devices,  simulators,  and 
simulations  (TADSS).  These  may  be  used  more  extensively  in  school 
or  at  soldiers'  units  and  home  stations  through  "distance  learning" 
technologies. 

The  Army  asked  the  Arroyo  Center  to  develop  and  apply  research 
methods  to  flesh  out  such  training  approaches,  determine  how  they  would 
be  implemented  in  practice,  and  assess  their  costs  and  effectiveness. 
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We  Developed  Systematic  Methods  to  Screen 
and  Evaluate  Training  Approaches 


Our  research  provided  tools  and  analyses  that,  if  applied  sequentially  and 
systematically,  constitute  an  integrated  method  for  evaluating  training 
approaches.  There  are  four  steps. 

1.  Analyze  all  Army  occupations,  determine  where  costs  could  be 
lowered  significantly  through  new  training  approaches,  and  select 
ones  with  high-payoff  potential.  An  example  of  this  involves  finding 
the  occupations  where  it  makes  most  sense  to  shift  training  to  unite  or 
to  expand  use  of  training  technologies. 

2.  For  specific  occupations,  collect  and  analyze  job  performance  measures 
(what  soldiers  actually  do),  and  compare  job  and  training  requirements 
to  current  curriculum.  Then,  design  alternative  courses  that  change 
training  content  and  methods  in  ways  that  preserve  training 
effectiveness.  An  example  of  this  involves  identifying  how  much 
material  is  really  suitable  for  training  in  units  and  laying  out  curricula 
and  technologies  to  support  it. 

3.  Analyze  resources  and  costs  of  changing  training  approaches  using  an 
"activity-based  costing"  method.  This  method  examines  major 
functions  of  training  development,  training  delivery,  and  training 
support;  determines  changes  in  resources  and  activities  needed  to 
change  training;  and  assigns  cost  factors  to  the  resource  changes.  It 
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also  determines  one-time  start-up  costs,  annual  recurring  costs  and 
savings,  and  the  time  to  break  even. 

4.  Take  the  alternatives  that  promise  to  save  costs,  and  if  there  are  doubts 
that  the  new  approach  will  preserve  training  effectiveness,  conduct 
randomized  clinical  trials  comparing  effects  of  alternative  training 
approaches  on  soldier  performance. 

Our  research  applied  elements  of  this  methodology  in  a  number  of 
occupations  and  training  courses,  covering  officers  and  enlisted,  combat 
and  civilian-like,  high-tech  and  low-tech,  over  such  areas  as  armor, 
artillery,  communications,  and  maintenance.  These  were  generally 
selected  as  test  cases  from  which  results  could  reasonably  generalize  to 
other  occupations  and  training  courses. 
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Research  Issues  and  Findings 


Issues  Findings 

Shifting  training  from  school  •  School  training  is  generally  well 
to  work  settings  aligned  with  fob  needs 

•  Major  shifts  to  units  can  raise 
costs 

•  Costs  and  savings  depend  on  need 
for  new  infrastructure  and  delivery 

_ method 


Expanding  use  of  training 
technologies 


As  mentioned  earlier,  one  general  approach  for  cutting  training  costs  and 
preserving  effectiveness  is  shifting  training  from  the  school  setting  to  the 
work  setting  by  increasing  the  amount  of  self-development  or  OJT. 

In  a  number  of  studies  exploring  the  potential  for  shrinking  the 
schoolhouse  and  expanding  training  in  units,  we  found: 

•  Army  techniques  for  designing  training  programs  to  support  job 
requirements  work  fairly  well; 

•  Attempts  to  shift  the  training  to  units  will  often  not  produce  the 
desired  outcome,  that  is,  lower  the  costs  of  training; 

•  The  amount  of  costs  and  savings  depends  on  the  need  for  new 
infrastructure  to  support  training  in  units  and  the  way  the  training  is 
delivered  in  units. 
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Most  Material  Trained  in  School 
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The  Army  pioneered  the  development  of  instructional  design  methods,  in 
particular  the  systems  approach  to  training,  which  sets  skill  standards  for 
occupationals  and  devises  curriculum  to  provide  the  necessary 
knowledge,  skills,  and  abilities  in  school  and  on  the  job.  Our  analysis 
comparing  job  requirements  to  current  training  approaches  found,  in 
general,  a  good  alignment  between  school  and  work  in  the  training 
courses  we  examined. 

We  examined  this  issue  in  three  courses:  the  armor  officer  advanced 
course  (AO AC),  which  prepares  armor  captains  to  be  company 
commanders;  the  introductory  course  for  enlisted  tank  crewmen;  and  the 
introductory  course  for  artillery  fire  direction  specialists,  who  receive 
target  data  from  forward  observers,  compute  angle  and  direction  of  fire, 
and  communicate  the  solution  to  the  artillery  batteries. 

The  above  graph  shows  how  much  of  current  schoolhouse  training  our  "job 
analysis"  indicated  was  "proper"  for  training  in  school. 

•  Most  material  needed  to  perform  the  job,  requiring  training  in  school, 
was  trained  in  school — 75  to  93  percent  of  current  course  hours  in  these 
cases. 
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•  Only  rarely  was  "unnecessary"  training  found  that  could  be  omitted 
(i.e.,  training  on  tasks  that  wouldn't  actually  be  performed  in  the 
subsequent  job). 

*  Some  material  could  be  shifted  to  units,  but  not  a  lot.  The  best 
example  was  in  the  AOAC.  But  even  here,  only  25  percent  of  the 
current  course  material  was  found  to  be  amenable  for  a  shift,  which 
was  less  than  had  been  estimated  under  the  Army's  plan  for 
distributed  training. 

Thus,  tiie  potential  is  limited  for  shifting  training  from  schools  and  into 
units — at  least  in  terms  of  the  quantity  of  material  amenable  to  such  a 
shift. 
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Shifting  Training  to  Units  Requires  Changes 
in  Training  Strategy  and  Resources 
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Potential  resource  implications 
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What  happens  when  attempts  are  made  to  move  training  out  of  school 
and  into  units  where  there  is  some  potential  for  doing  so?  This  example 
comes  from  our  analysis  of  "distance  learning"  for  armor  officers.  Here 
we  examined  the  effects  of  moving  four  weeks  of  training  from  the  school 
to  the  unit  by  having  officers  engage  in  "self-study"  prior  to  attending  the 
armor  officer  advanced  course.  This  training  would  be  supported  with 
media  like  computer-based  training,  teletraining,  and  traditional  paper 
products. 

Our  research  found  that  the  costs  and  savings  of  this  program  depend  on 
the  capacity  of  units  to  absorb  this  additional  training,  that  is,  whether 
courseware  and  equipment  already  exist,  whether  facilities  like  post 
learning  centers  are  adequate,  whether  additional  manpower  would  be 
needed  to  oversee  the  training,  and  so  forth. 
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Shifting  Training  to  Units  Appears  to  Pay  Off 
If  Unit  Resources  Are  Adequate 


8.0 


One-time  cost  5-year  savings 


If  one  were  willing  to  assume  that  units  had  the  capacity  to  absorb  this 
training — the  assumption  initially  favored  by  the  Army's  Training  and 
Doctrine  Command — then  shifting  training  to  self-study  in  units  could 
lower  training  costs.  The  graph  above  shows  RAND's  analysis  of  costs 
and  savings  under  this  assumption.  For  a  one-time  cost  of  $1.7  million, 
savings  would  accrue  quickly,  totaling  $8.0  million  by  the  end  of  a  five- 
year  period.  This  represents  a  breakeven  point  of  about  one  year. 
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Cost  and  Savings  Depend  on  Whether 
New  Infrastructure  Is  Needed 


Training 

Resources/ 

Assumptions  | 
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Low  Cost 

High  Cost 

Delivery 

Facilities  and 
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Logistics  and 
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But  as  indicated,  this  analysis  assumes  this  new  training  to  be  within  the 
capacity  of  units  to  deliver.  Thus,  they  are  "low  cost"  assumptions 
because  they  require  no  new  investments  in  infrastructure. 

There  is  a  contrasting  set  of  assumptions,  which  we  term  "high  cost" 
assumptions.  The  above  table  shows  the  difference  between  the  "low 
cost"  and  "high  cost"  assumptions  used  in  our  analysis. 

•  The  major  training  functions  examined  in  our  cost  analysis  are  shown 
on  the  left — delivering  training,  developing  training  products,  and 
"overhead"  support  for  the  training. 

•  Next  are  the  major  resources  and  activities  included  in  each  of  these 
functions  (e.g.,  delivering  training  requires  facilities,  equipment, 
people,  and  time  to  conduct  it,  products  to  support  it,  and  ongoing 
distribution  and  maintenance). 

•  The  final  two  columns  show  how  these  resources  and  activities  are 
treated  under  the  "low  cost"  and  "high  cost"  assumptions.  The 
assumptions  about  costs  of  training  delivery  are  the  most  important. 

In  contrast  to  the  low  cost  assumptions,  the  high  cost  assumptions  state 
that  the  program  would  need  additional  manpower  to  support  training 
in  units,  new  equipment  to  support  the  training,  remodeled  facilities 
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(such  as  learning  centers)  to  hold  additional  training  material,  and 
other  activities  (e.g.,  for  development  and  support). 
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Payoff  Is  Much  Smaller  Under 
High  Cost  Assumptions 


Low  cost  assumptions 
8-  _ 8-0 


$  millions 


One-time  5-year 
cost  savings 


High  cost  assumptions 
7.2 


One-time  5-year 
cost  savings 


Based  on  our  analysis,  we  believed  the  "high  cost"  assumptions  were 
more  likely.  Our  analyses  showed  that  if  the  high-cost  assumptions  were 
correct,  the  initial  start-up  costs  would  increase,  while  the  recurring 
savings  would  decrease  considerably. 

•  The  left  panel  in  the  above  graph  shows  the  cost  results  under  the  low 
cost  assumptions  from  earlier.  The  right  panel  shows  how  the  cost 
results  change  under  the  high  cost  assumptions. 

•  Now,  instead  of  returning  $8  million  on  a  $1.7  million  investment,  it 
would  take  an  investment  of  $7.2  million  to  get  $4  million  back  in  five 
years. 

•  The  breakeven  point  goes  from  one  year  to  about  nine,  which  is  not 
nearly  as  good  an  economic  proposition. 

All  in  all,  this  analysis  shows  that  shifting  training  from  the  schoolhouse 
to  self-study  in  units  won't  necessarily  produce  the  desired  outcome, 
lowering  the  costs  of  training.  If  you  believe  the  higher  cost  assumptions, 
a  large  up-front  investment  is  required  that  will  not  pay  off  for  a  long 
time — perhaps  longer  than  the  currency  of  the  material  the  new  training  is 
designed  to  deliver. 
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Payoff  Is  Sensitive  to  Training  Delivery  Methods 


•  Compared  three  media  mixes  for  delivering  training 

-  Computer-based  training,  with  paper  and 
videotape 

-Teietraining  plus  other  media 
-100%  paper 

•  Teietraining  does  not  pay  off  for  this  course 

-  Large  start-up  costs 

-Yearly  operating  costs  exceed  those  of  current 
course 


There  is  a  further  wrinkle  to  these  findings — the  amount  of  costs  and 
savings  is  also  sensitive  to  the  mix  of  media  used  to  deliver  the  training. 
The  previous  analysis  used  the  first  set  of  media  shown  on  the  above 
chart,  a  mixture  of  computer-based  training  (CBT),  paper,  and  videotape. 
We  examined  two  additional  alternatives — a  more  complex,  "high  tech" 
mix  that  included  videoteletraining,  and  a  less  complex,  "low  tech"  mix 
that  included  only  paper  (e.g.,  reading  material,  worksheets,  and  so  forth). 

The  analyses  showed  that  some  media  can  actually  increase  the  costs  of 
the  training,  compared  to  current  methods.  For  example,  using 
teietraining  to  support  training  in  units  would  cost  more  each  year  to 
operate  than  current  training — under  any  set  of  assumptions.  The  recurring 
costs  increase  because  as  the  media  become  more  complex,  operation  and 
maintenance  costs  increase. 

In  general,  as  distance  learning  media  become  more  complex,  the  one¬ 
time,  start-up  costs  increase,  while  the  recurring  savings  decrease. 
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Research  Issues  and  Findings  | 

Issues 

Findings 

Shifting  training  from  school 
to  work  settings 

Expanding  use  of  training 
technologies 

•  Considerable  room  to  increase 
use  of  technology  in  schools 

•  Most  savings  obtained  by 
adapting  existing  resources 

•  Technology  should  be  used  to 
replace,  not  enhance,  hands-on 
training 

The  second  set  of  findings  addresses  whether  expanding  the  use  of 
training  technologies  can  reduce  costs  but  preserve  effectiveness.  Our 
findings  show: 

•  There  is  room  to  expand  the  use  of  technology  to  deliver  training  in 
schools,  and  doing  so  can  lower  the  costs  of  training; 

•  From  the  cost  perspective,  it  is  better  to  adapt  existing  technologies, 
using  material  that  is  already  on  hand  or  available  off-the-shelf,  rather 
than  build  new  technologies  from  scratch; 

•  The  effectiveness  of  training  technology  depends  on  how  it  is  used — that 
is,  there  is  a  bigger  payoff  from  using  technology  as  a  substitute  for 
more  costly  training  methods  than  there  is  to  "enrich"  training — the 
more  common  use  of  training  technology. 
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Use  of  Training  Technologies  Can 
Be  Expanded  Within  Schools 


Percent  of  Armor  Crewman  Course 
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Potential  Uses  of  Training  Devices 

•  Turret  trainers:  Develop 
loading  and  gunnery  skills 

•  Hull  driver  trainer:  Practice 
driving,  towing,  and  recovery 

•  Driver  trainer  mock-up: 
Familiarize  with  driver  station 

•  Computer-based  training: 
Replace  lectures  and  practice 
troubleshooting 


Our  methodology  for  analyzing  and  developing  alternatives  to  current 
training  courses  was  also  used  to  examine  alternative  uses  of  resources  to 
train  in  the  schoolhouse.  The  aim  was  to  determine  where  use  of  TADSS 
could  be  feasible.  From  this,  we  found  that  there  was  considerable  room 
to  expand  the  use  of  training  technologies  within  schools. 

This  analysis  used  principles  of  instructional  systems  design  to  suggest 
candidates  for  training  technology.  The  method  matches  tasks  to 
technologies  and  suggests  alternatives,  considering  criteria  like  skill 
requirements,  need  for  instructor  supervision,  and  equipment  cost. 

We  did  this  in  a  number  of  courses,  but  we  illustrate  the  approach  with 
the  example  of  the  Abrams  armor  crewman  course,  which  trains  a  new 
enlisted  soldier  to  serve  as  a  tank  crewman.  When  we  examined  this 
course,  we  found  many  of  the  tasks  now  trained  using  tanks  could  be  trained 
using  training  devices.  The  left  bar  in  the  above  figure  shows  die  current 
course,  in  which  there  is  minimal  use  of  training  technology — in  this  case, 
a  driver  simulator  for  11  percent  of  current  course  hours.  The  right  bar 
shows  an  alternative,  in  which  the  use  of  tanks  decreases  from  68  percent 
to  20  percent  of  current  course  hours,  while  the  use  of  training  devices  and 
simulators  increases  from  11  percent  to  59  percent  of  current  course  hours. 

Note  that  the  training  technologies  do  not  replace  the  tank  entirely;  instead, 
their  relative  usage  changes.  For  example,  the  analysis  examines  the  use 
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of  turret  trainers  to  learn  and  practice  procedures  performed  by  the 
loader,  gunner,  and  so  forth. 
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But  Savings  Depend  on  How  Training 
Devices  Are  Obtained 


Our  cost  analysis  showed  that  use  of  training  technologies  in  this  fashion 
could  reduce  costs,  but  the  amount  of  costs  and  savings  depended  on  how 
the  devices  were  obtained  (as  shown  in  the  above  figure).  If  they  were 
built  from  scratch — say  acquired  from  a  defense  contractor — we  estimated 
that  start-up  costs  of  $20  million  would  return  $16  million  in  the  initial 
five-year  period.  The  payback  period  is  approximately  six  years.  Given 
the  lifecycle  of  the  Abrams  tank,  this  is  probably  a  worthwhile  investment. 

But  if  the  devices  are  adapted  from  existing  tanks,  by  converting  the  tanks 
in  inventory  at  Fort  Knox  into  turret  trainers  and  hull  driver  trainer 
vehicles,  start-up  costs  of  $8  million  would  return  $38  million  in  savings  in 
five  years,  representing  savings  in  tank  operating  costs  and  maintenance. 
This  program  would  take  two  years  to  break  even. 

Incidentally,  converting  tanks  to  training  devices  is  not  as  radical  an  idea 
as  it  may  appear.  This  was  done  with  the  earlier-generation  M60  tanks. 
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— 
Clinical  Trials  Examined  Effects  of 
Technology  on  Proficiency 


•  Experiments  conducted  in  communications  courses 

-  Multichannel  communications  equipment 
-Tactical  satellite/microwave  systems 

•  Experiments  used  random  assignment,  monitoring  of 
training,  and  hands-on  performance  tests 

•  Contrasting  strategies  tested  in  two  experiments 

-  Enhance  training  opportunity 

-  Substitute  for  more  expensive  hands-on  training 


The  cost  analysis  of  armor  training  devices  shows  that  use  of  training 
devices  can  save  money.  The  question  then  is  how  effective  they  will  be. 
Clinical  trials  comparing  technology-based  training  to  current  methods  of 
training  are  needed  to  answer  this  question. 

We  were  not  able  to  assess  the  effectiveness  of  training  technologies  for 
the  armor  crewman  (because  they  have  not  yet  been  built),  but  we  did 
examine  this  in  another  series  of  studies  conducted  in  Army  training 
classrooms.  These  studies  showed  that  the  effectiveness  of  training 
technologies  depends  on  how  they  are  used. 

The  experiments  were  conducted  in  communications  courses  at  the  Army 
Signal  school,  in  initial  training  courses  for  two  "high-tech"  occupations: 
the  multichannel  communications  equipment  operator  course,  and  the 
tactical  satellite/microwave  systems  operator  course. 

The  experiments  employed  classical  principles:  random  assignment  of 
students  to  control  and  experimental  classrooms,  measurement  of  training 
received  under  alternative  training  methods,  and  measurement  of  student 
performance  using  hands-on  tests. 

These  studies  examined  two  contrasting  strategies  for  using  training 
technology.  One  is  to  enhance  training  by  adding  technologies  to  existing 
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resources.  The  other  is  to  substitute  for  hands-on  training  in  a  resource 
mix,  as  in  the  armor  crewman  example. 
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Enhancement  Experiment:  Using  Interactive 
Videodisc  (IVD)  to  Increase  Training  Opportunity 

Multichannel  Communications  Equipment  Course 

Control  Classroom 
(Average  25  students) 

Experimental  Classroom  1 

(Average  25  students)  1 

y  y  y  y  y 
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The  above  chart  illustrates  how  we  conducted  the  enhancement 
experiment.  The  "control"  classroom  is  the  current  situation.  The 
classroom  contains  ten  multichannel  radio  sets  and  serves  25  students. 
There  is  an  appearance  of  inefficiency  because  10  students  can  train  on  the 
radios  while  the  remainder  work  at  desks  waiting  for  a  hands-on  training 
opportunity. 

Signal  school  personnel  had  the  idea  that  they  could  increase  practice 
opportunities  by  obtaining  and  developing  simulations  on  interactive 
videodisc  (IVD).  IVD  combines  computer-assisted  instruction  on  a  PC 
with  visual  images  from  a  videodisc.  Here  it  was  used  to  simulate  the 
procedures  for  operating  the  radio.  The  trainee  uses  a  touch  pen  to  place 
cables  where  they  belong,  set  meters,  and  so  forth. 

The  experiment  tested  this  idea — enhance  training  opportunity  by  adding 
IVD  simulations  to  the  existing  classroom.  We  persuaded  the  Signal 
school  to  set  up  side-by-side  classrooms  that  looked  like  this  and  to 
randomly  assign  students  to  one  or  the  other  for  training.  We  later 
measured  the  students'  performance  using  hands-on  tests. 

Over  200  students  participated  in  this  experiment. 
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Results  of  Enhancement  Experiment  | 


•  IVD  program  was  fully  accepted  and  implemented 
in  classrooms 

•  Practice  time  increased  by  45%  within  same  course 
length 

•  But  enhancement  produced  no  significant 
improvement  in  performance 

-  Performance  was  very  high  to  begin  with 


The  results  were  a  bit  disappointing.  The  IVDs  were  well-accepted  by 
instructors  and  students,  and  they  were  used  to  increase  practice  time. 

The  IVD  group  received  nearly  half  again  as  much  training  time  as  the 
control  group.  But  performance  did  not  improve  that  much.  In  a  later  test 
of  performance,  the  IVD  group  was  able  to  complete  the  task  faster,  with 
less  effort  and  fewer  mistakes.  But  both  groups  were  very  proficient  at 
meeting  the  military  standard:  96  percent  of  the  trainees  eventually  got 
the  task  right  in  the  allotted  time.  Proficiency  seemed  adequate  to  begin 
with. 

It  seems  that  initial  proficiency  should  have  been  taken  into  account 
before  adding  resources  to  classrooms.  Even  though  there  were  only  10 
radios  and  25  students,  practice  time  was  adequate. 
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Substitution  Experiment:  Using  IVD  in  Place 
of  More  Expensive  Equipment 


Tactical  Satellite/Microwave  Systems  Course 
Control  Classroom  Experimental  Classroom 


Cost  of  equipment  is  five  times  greater  in  control  classroom 


This  experience  led  us  to  recommend  another  strategy  for  using  training 
technology — to  reduce  training  costs  and  preserve  effectiveness  by 
substituting  technology  for  hands-on  training  where  technology  was 
cheaper.  We  proposed  another  experiment  to  the  Signal  school,  which 
changed  the  mix  of  resources  used  to  train  by  substituting  IVDs  for  some 
(but  not  all)  of  the  radios. 

The  experiment  is  illustrated  in  the  chart  above.  We  again  set  up  side-by- 
side  classrooms,  this  time  in  a  different  training  course.  The  "control" 
group  trained  in  the  standard  method,  using  a  room  full  of  tropospheric 
scatter  radios.  The  experimental  group  used  only  a  couple  of  radios, 
relying  mostly  on  IVD  simulations. 

The  notable  feature  here  is  that  the  experimental  classroom  is  much  less 
costly  than  the  control  classroom — by  a  factor  of  five  in  the  costs  of  the 
radio  versus  the  radio /IVD  mixture.  It  also  turned  out  that  running  the 
radio /IVD  classroom  required  fewer  instructors,  by  a  factor  of  two,  since 
less  over-the-shoulder  supervision  was  needed  to  use  the  radio/IVD  mix. 

Over  200  students  participated  in  this  experiment. 
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Substitution  Allowed  Nearly  Equivalent 
Training  Opportunities 


Experimental 
Classroom 
(Mix  of  radios/ 
IVD) 


Hands-on 

Training  IVD  Training 
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Despite  the  resource  substitution,  the  students  received  roughly 
equivalent  training  in  the  two  conditions.  The  above  graph  shows  the 
number  of  "practice  sessions"  received  by  the  students  in  the  two 
classrooms.  Each  practice  session  covered  a  number  of  procedures  needed 
to  install  and  operate  tropospheric  scatter  radios.  Students  in  the  control 
classroom  received  about  15  sessions  in  which  to  practice  the  tasks. 
Students  in  the  experimental  classroom  received  14  sessions  on  average  in 
which  to  practice  the  task,  9  of  which  occurred  on  IVD. 
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Performance  Remained  Constant 
Despite  Substitution 
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The  above  graph  shows  how  the  students  in  the  two  conditions  performed 
on  hands-on  tests  of  several  tasks  covered  in  the  practice  sessions  using 
radios  or  the  radio/TVD  mixture. 

The  two  groups  were  statistically  indistinguishable.  Both  groups  were 
equally  able  to  perform  die  tasks  to  Army  standard.  Performance  was  not 
all  that  high  in  all  cases,  but  it  did  not  matter  whether  the  students  were 
trained  using  equipment  or  a  mix  of  equipment  and  simulation. 
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Army  Uses  of  RAND  Research  | 

Policy  changes 

•  Army  plans  to  develop  tank  training 
devices  per  RAND  recommendations 

•  Army  cancelled  $180  million  budget 
item  for  teletraining  equipment 

•  Training  courses  restructured  along 
lines  recommended  by  RAND 

New  methods 

•  Approach  for  overhauling  training  courses 

adopted 

•  Method  for  costing  training  changes 

•  Technique  for  setting  funding  priorities  for 
interactive  courseware 

Collectively,  the  above  studies  have  made  a  number  of  specific 

recommendations  about  how  to  restructure  courses,  invest  in  technology, 

and  use  training  resources  most  effectively. 

The  Army  has  adopted  a  number  of  our  specific  recommendations. 

•  The  Army  plans  to  convert  tanks  to  tank  training  devices. 

•  The  Army  dropped  from  budget  planning  documents  an  item  for 
teletraining  equipment  totaling  $180  million. 

•  Specific  courses,  like  the  artillery  fire  direction  specialist  and  others, 
were  restructured  based  on  RAND  research  results. 

In  addition,  the  Army  has  adapted  and  implemented  several  of  our 

methodologies. 

•  The  approach  for  examining  occupations  and  reviewing  job 
requirements  and  current  training  approaches  has  been  incorporated 
as  part  of  a  new  Army  occupational  training  system. 

•  The  methodology  developed  for  analyzing  training  resources  and  costs 
has  been  incorporated  in  Army  professional  education  courses  for 
training  developers  and  managers. 
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•  Criteria  establishing  appropriate  uses  of  training  technology,  drawn 
from  our  work,  were  used  to  set  priorities  for  allocating  funding  for 
IVD  courseware  development. 
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Lessons  from  These  Studies  | 


•  Ensure  proper  alignment  between  school  and  work 

-  Set  skill  standards  and  measure  job  performance 

-  Use  job  performance  data  to  develop  and  revise  curricula 

•  Develop  an  Investment  plan  for  training  technology 

-  Emphasize  existing  technologies,  adapted  as  needed 

-  Add  resources  only  where  performance  is  deficient 

-  Substitute  technology  in  place  of  more  expensive  training 
resources 

•  Experiment  and  monitor  new  approaches 

-  Assess  training  outcomes  and  payoffs 

-  Verify  cost  parameters  and  assumptions 


We  have  also  drawn  some  general  lessons  from  our  Army  training 
research. 

First,  the  research  has  shown  the  value  in  the  Army's  approach  to 
designing  instructional  systems.  The  system  is  particularly  strong  at 
identifying  occupational  skill  standards  and  "translating"  them  into  a 
curriculum  that  meets  job  skill  requirements. 

But  to  the  extent  that  the  system  could  be  strengthened,  it  is  in  the  area  of 
routine  monitoring  and  feedback.  Hence,  we  have  also  learned  that  to 
keep  training  in  alignment  with  job  requirements,  it  is  necessary  to  collect 
data  on  work  performed  on  the  job  and  to  feed  this  information  back  on  a 
routine  basis  to  ensure  that  training  remains  current  with  job  requirements 
and  uses  the  most  efficient  resources. 

Second,  we  have  emphasized  the  need  for  a  systematic,  coordinated 
investment  plan  for  technology.  The  plan  should  seek,  as  much  as 
possible,  to  use  on-hand  and  off-the-shelf  technologies,  such  as  videotapes 
and  CBT  on  personal  computers,  instead  of  one-of-a-kind,  complex 
systems.  This  is  not  to  preclude  some  expensive  technologies,  but  it 
establishes  priorities  where  performance  needs  improvement,  while 
seeking  to  maximize  the  use  of  technology  to  improve  efficiency  and 
reduce  costs. 
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Finally,  we  have  observed  the  value  of  field  trials  and  tests  of  new  training 
methods,  which  provide  direct  evidence  of  the  costs  and  effectiveness  of 
new  training  approaches. 
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How  Applicable  Are  These  Lessons  for 
Civilian  Education  and  Training? _ 

•  Civilian  education/training  system  different  from  military 

-  Employers  and  educators  not  as  well  connected 

-  Serves  broader  population  with  wider  needs 

-  Key  problem  is  to  increase  effectiveness 

•  But  lessons  may  transfer,  especially  in  technical  fields 

-  Occupational  skills  standards  and  performance 
measures  for  comparable  jobs 

-  Instructional  design  methods  for  aligning  training  with 
work 

-  Experience  with  educational  courseware  and 
simulations 

Key  need:  Rigorous  assessments  of  benefits  and  costs 


The  above  research  findings  and  lessons  have  been  developed  from 
research  on  Army  training  courses.  We  feel,  however,  that  insights  of 
these  studies  could  help  civilian  education  and  training  institutions 
improve  the  preparation  of  citizens  for  work. 

Obviously,  there  are  differences  between  the  military  and  the  civilian 
education  and  training  system.  The  military  is  a  "closed  system"  in  which 
employers  and  educators  belong  to  the  same  organization.  Military 
personnel  are  a  select  group,  nearly  all  with  high  school  diplomas,  most 
scoring  high  on  aptitude  tests  for  the  jobs  in  which  they  are  trained.  They 
are  volunteers  with  high  motivation  to  learn  the  job  and  a  contractual 
obligation  to  do  so.  The  civilian  world  is  different,  with  employers  and 
educators  often  at  loggerheads,  and  there  is  more  variability  in 
educational  objectives  and  populations  served. 

Moreover,  where  the  military's  problem  is  to  reduce  the  costs  of  training 
while  preserving  its  effectiveness,  the  problem  in  civilian  settings  is  often 
that  of  increasing  effectiveness  within  available  resources. 

Despite  these  differences,  we  think  there  are  enough  essential  points  of 
similarity  to  permit  leveraging  the  military  experience  for  civilian 
purposes.  First,  the  military  experience  emphasizes  the  importance  of 
having  occupational  skills  standards  and  measures  of  performance  that 
establish  what  people  need  to  know  to  perform  effectively  on  the  job. 
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Indeed,  many  existing  military  skill  standards  and  performance  measures 
may  transfer  to  similar  civilian  jobs. 

Second,  die  military  experience  also  demonstrates  the  importance  of 
systematically  and  routinely  linking  the  standards  to  curriculum. 
Principles  of  the  military's  instructional  design  methods  might  usefully  be 
applied  in  civilian  settings. 

Third,  the  military  experience  points  to  the  need  for  a  hard-nosed, 
practical  approach  to  developing  technology  and  applying  resources  in 
support  of  that.  Military  experience  with  educational  technology 
(hardware  and  courseware)  and  its  applications  (e.g.,  for  simulation)  is 
informative. 

Finally,  and  perhaps  most  important,  the  military  experience  shows  the 
need  and  the  value  of  attending  carefully  to  costs  and  benefits  and  of 
conducting  systematic  assessments  of  education  and  training  innovations. 
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